The present invention relates to an image process device, an image process method, and an image process program, for executing a process of converting a 2D image into a 3D image for stereoscopic vision.
In recent years, 3D video content items such as 3D movies or 3D broadcasting have been widespread. In order to allow an observer to perform stereoscopic vision, a right eye image and a left eye image with parallax are required. When a 3D video is displayed, a right eye image and a left eye image are displayed in a time-division manner, and the right eye image and the left eye image are separated using glasses for video separation such as shutter glasses or polarization glasses. Thereby, an observer can perform stereoscopic vision by observing the right eye image only with the right eye and the left eye image only with the left eye. In addition, if a right eye image and a left eye image are not temporally divided but spatially divided, glasses are not necessary but a resolution is reduced. In any of the glasses method and the glassless method, a right eye image and a left eye image are commonly necessary.
There are largely two methods of producing 3D images, that is, one is a method of simultaneously capturing a right eye image and a left eye image using two cameras, and the other is a method of generating a parallax image by editing a 2D image captured by a single camera afterward. The present invention relates to the latter and relates to a 2D-3D conversion technique.
FIG. 1 is a diagram illustrating a basic process procedure of 2D-3D conversion. First, a depth map (also referred to as depth information) is generated from a 2D input image (step S10). In addition, 3D images are generated using the 2D input image and the depth map (step S30). In FIG. 1, the 2D input image is set as a right eye image of the 3D output images, and an image obtained by shifting pixels of the 2D input image using the depth map is set as a left eye image of the 3D output images. Hereinafter, a set of a right eye image and a left eye image with predetermined parallax is referred to as 3D images or parallax images.
If 3D images are to be generated, pixels of a 2D image are shifted using a depth map, and a 2D image of a different viewpoint with parallax for the 2D image is generated. Omitted pixels occur in the generated 2D image of the different viewpoint due to the pixel shift. Generally, the omitted pixels are interpolated from peripheral pixels.    Patent Literature 1: Japanese Patent Application Laid-Open No. 2011-48586    Patent Literature 2: Japanese Patent Application Laid-Open No. 2009-44722
In a case where a step-difference in depths at an object boundary is large inside a screen, a pixel shift amount of the boundary part increases. Therefore, the number of omitted pixels, that is, the area of an omitted region also increases. As described above, the omitted pixels are interpolated from peripheral pixels; however, if the area of the omitted region increases, a location where an interpolated pixel does not match an interpolation position tends to occur.